Of Measurement Uses, and Consequently It Should Coefficient of Concordance

be viewed within a much larger system of reliability analysis, generalizability theory. Moreover, alpha focused attention on reliability coefficients when that attention should instead be cast on measurement error and the standard error of measurement. For Cronbach, the extension of alpha (and classical test theory) came when Fisherian notions of experimental design and analysis of variance were put together with the idea that some ‘‘treatment’’ conditions could be considered random samples from a large universe, as alpha assumes about item sampling. Measurement data, then, could be collected in complex designs with multiple variables (e.g., items, occasions, and rater effects) and analyzed with random-effects analysis of variance models. The goal was not so much to estimate a reliability coefficient as to estimate the components of variance that arose from multiple variables and their interactions in order to account for observed score variance. This approach of partitioning effects into their variance components provides information as to the magnitude of each of the multiple sources of error and a standard error of measurement, as well as an ‘‘alpha-like’’ reliability coefficient for complex measurement designs. Moreover, the variance-component approach can provide the value of ‘‘alpha’’ expected by increasing or decreasing the number of items (or raters or occasions) like those in the test. In addition, the proportion of observed score variance attributable to variance in item difficulty (or, for example, rater stringency) may also be computed, which is especially important to contemporary testing programs that seek to determine whether examinees have achieved an absolute, rather than relative, level of proficiency. Once these possibilities were envisioned, coefficient alpha morphed into generalizability theory, with sophisticated analyses involving crossed and nested designs with random and fixed variables (facets) producing variance components for multiple measurement facets such as raters and testing occasions so as to provide a complex standard error of measurement. By all accounts, coefficient alpha—Cronbach’s alpha—has been and will continue to be the most popular method for estimating behavioral measurement reliability. As of 2004, the 1951 coefficient alpha article had been cited in more than 5,000 publications.